{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "e6191e1c",
   "metadata": {},
   "source": [
    "**NOTE:** This notebook was written in 2024, and is not guaranteed to work with the latest version of llama-index. It is presented here for reference only."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "42c463c6-a36e-41ed-9897-0b7b25417deb",
   "metadata": {},
   "source": [
    "![Slide One](https://d3ddy8balm3goa.cloudfront.net/genai-philippines/v1/1.svg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "08f96a7c-6854-4421-ae55-7206ca265382",
   "metadata": {},
   "source": [
    "![Divider Image](https://d3ddy8balm3goa.cloudfront.net/mlops-rag-workshop/divider-2.excalidraw.svg)\n",
    "![Slide Two](https://d3ddy8balm3goa.cloudfront.net/genai-philippines/v1/2.svg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4bf4cac4-4a35-4eb1-a2a2-39388ee69030",
   "metadata": {},
   "source": [
    "![Divider Image](https://d3ddy8balm3goa.cloudfront.net/mlops-rag-workshop/divider-2.excalidraw.svg)\n",
    "![Slide Three](https://d3ddy8balm3goa.cloudfront.net/genai-philippines/v1/3.svg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a0eace5e-b710-4879-822b-2fe88257bec2",
   "metadata": {},
   "source": [
    "![Divider Image](https://d3ddy8balm3goa.cloudfront.net/mlops-rag-workshop/divider-2.excalidraw.svg)\n",
    "![Slide Four](https://d3ddy8balm3goa.cloudfront.net/vector-oss-tools/draft/4.svg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "86b21a33-f05f-4cb0-81a4-1d7b26d4ed10",
   "metadata": {},
   "source": [
    "## Observability: Arize AI\n",
    "\n",
    "Follow the quickstart guide found [here](https://github.com/Arize-ai/openinference/tree/main/python/instrumentation/openinference-instrumentation-llama-index#quickstart)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d961c72b-12ac-4134-b801-093f61f2766f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip available: \u001b[0m\u001b[31;49m22.3.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.1\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpython3.11 -m pip install --upgrade pip\u001b[0m\n",
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "%pip install --upgrade \\\n",
    "    openinference-instrumentation-llama-index \\\n",
    "    opentelemetry-sdk \\\n",
    "    opentelemetry-exporter-otlp \\\n",
    "    \"opentelemetry-proto>=1.12.0\" \\\n",
    "    arize-phoenix -q"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "524c697e-4b8d-4893-a6eb-1278e8d5143e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import os\n",
    "\n",
    "get_ipython().system = os.system\n",
    "\n",
    "!python -m phoenix.server.main serve > arize.log 2>&1 &"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "69298cc6-c951-45f9-addb-3203d14e6191",
   "metadata": {},
   "outputs": [],
   "source": [
    "from openinference.instrumentation.llama_index import LlamaIndexInstrumentor\n",
    "from opentelemetry.exporter.otlp.proto.http.trace_exporter import (\n",
    "    OTLPSpanExporter,\n",
    ")\n",
    "from opentelemetry.sdk import trace as trace_sdk\n",
    "from opentelemetry.sdk.trace.export import SimpleSpanProcessor\n",
    "\n",
    "endpoint = \"http://127.0.0.1:6006/v1/traces\"\n",
    "tracer_provider = trace_sdk.TracerProvider()\n",
    "tracer_provider.add_span_processor(\n",
    "    SimpleSpanProcessor(OTLPSpanExporter(endpoint))\n",
    ")\n",
    "\n",
    "LlamaIndexInstrumentor().instrument(tracer_provider=tracer_provider)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "00bee0d1-4056-44c9-ac0c-6652c178f5be",
   "metadata": {},
   "source": [
    "Now open a web browser and enter the url `http://localhost:6006/`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20ff8c48-a64c-4dea-b54b-6c46fc7b6bec",
   "metadata": {},
   "source": [
    "## Example: A Gang of LLMs Tell A Story"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a450e92a-de33-44c3-854e-05ed1ef31b3c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# INSTALL LLM INTEGRATION PACKAGES\n",
    "%pip install llama-index-llms-openai -q\n",
    "%pip install llama-index-llms-cohere -q\n",
    "%pip install llama-index-llms-anthropic -q\n",
    "%pip install llama-index-llms-mistralai -q\n",
    "%pip install llama-index-vector-stores-qdrant -q\n",
    "%pip install llama-index-agent-openai -q\n",
    "%pip install llama-index-agent-introspective -q\n",
    "%pip install google-api-python-client -q\n",
    "%pip install llama-index-program-openai -q\n",
    "%pip install llama-index-readers-file -q\n",
    "\n",
    "# INSTALL OTHER DEPS\n",
    "%pip install pyvis -q"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3b29f3ca-9ac2-4fd4-80cf-d5e3e2d4f83d",
   "metadata": {},
   "outputs": [],
   "source": [
    "import nest_asyncio\n",
    "\n",
    "nest_asyncio.apply()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "74fd3581-e029-45f9-805d-6d3b07cf8651",
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index.llms.anthropic import Anthropic\n",
    "from llama_index.llms.cohere import Cohere\n",
    "from llama_index.llms.mistralai import MistralAI\n",
    "from llama_index.llms.openai import OpenAI\n",
    "\n",
    "anthropic_llm = Anthropic(model=\"claude-3-opus-20240229\")\n",
    "cohere_llm = Cohere(model=\"command\")\n",
    "mistral_llm = MistralAI(model=\"mistral-large-latest\")\n",
    "openai_llm = OpenAI(model=\"gpt-4o\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "15685a8e-6efd-4f38-9fed-a2f8f4a0c5a0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "In a world where pizza toppings knew no bounds, one daring chef created a masterpiece: the \"Everything But The...\n"
     ]
    }
   ],
   "source": [
    "theme = \"over-the-top pizza toppings\"\n",
    "start = anthropic_llm.complete(\n",
    "    f\"Please start a random story around {theme}. Limit your response to 20 words.\"\n",
    ")\n",
    "print(start)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e41ddbed-d684-43a5-838e-6d0c90fa9942",
   "metadata": {},
   "outputs": [],
   "source": [
    "middle = cohere_llm.complete(\n",
    "    f\"Please continue the provided story. Limit your response to 20 words.\\n\\n {start.text}\"\n",
    ")\n",
    "climax = mistral_llm.complete(\n",
    "    f\"Please continue the attached story. Your part is the climax of the story, so make it exciting! Limit your response to 20 words.\\n\\n {start.text + middle.text}\"\n",
    ")\n",
    "ending = openai_llm.complete(\n",
    "    f\"Please continue the attached story. Your part is the end of the story, so wrap it up! Limit your response to 20 words.\\n\\n {start.text + middle.text + climax.text}\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "15139945-c25b-49f3-b00b-d8d8515f1812",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "In a world where pizza toppings knew no bounds, one daring chef created a masterpiece: the \"Everything But The...\n",
      "\n",
      " ...Nay, Even The Kitchen Sink\" pizza. His culinary madness challenged conventions, yielding a legendary creation. \n",
      "\n",
      "Would you like me to take the story in a specific direction? \n",
      "\n",
      "Suddenly, the pizza came alive, its toppings swirling into a vortex, revealing a portal to a new, delicious dimension.\n",
      "\n",
      "The chef stepped through, greeted by sentient toppings who crowned him their king, forever uniting worlds through culinary magic.\n"
     ]
    }
   ],
   "source": [
    "# let's see our story!\n",
    "print(f\"{start}\\n\\n{middle}\\n\\n{climax}\\n\\n{ending}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c4e2ea2d-9505-4a65-bb2c-1b21671e5f6e",
   "metadata": {},
   "source": [
    "![Divider Image](https://d3ddy8balm3goa.cloudfront.net/mlops-rag-workshop/divider-2.excalidraw.svg)\n",
    "![Slide Five](https://d3ddy8balm3goa.cloudfront.net/genai-philippines/v1/5.svg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc1cb754-ea9a-4144-97a4-a08460b8ee9a",
   "metadata": {},
   "source": [
    "![Divider Image](https://d3ddy8balm3goa.cloudfront.net/mlops-rag-workshop/divider-2.excalidraw.svg)\n",
    "![Slide Six](https://d3ddy8balm3goa.cloudfront.net/genai-philippines/v1/6.svg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "229885d5-70d3-41c7-a156-f3a0874f7576",
   "metadata": {},
   "source": [
    "![Divider Image](https://d3ddy8balm3goa.cloudfront.net/mlops-rag-workshop/divider-2.excalidraw.svg)\n",
    "![Slide Seven](https://d3ddy8balm3goa.cloudfront.net/genai-philippines/v1/7.svg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a9acc43-7e6e-40e6-9bcb-cf695f718baf",
   "metadata": {},
   "source": [
    "![Divider Image](https://d3ddy8balm3goa.cloudfront.net/mlops-rag-workshop/divider-2.excalidraw.svg)\n",
    "![Slide Eight](https://d3ddy8balm3goa.cloudfront.net/genai-philippines/v1/8.svg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0d32def2-75ce-46b4-aef9-adbd21a47e58",
   "metadata": {},
   "source": [
    "![Divider Image](https://d3ddy8balm3goa.cloudfront.net/mlops-rag-workshop/divider-2.excalidraw.svg)\n",
    "![Slide Nine](https://d3ddy8balm3goa.cloudfront.net/genai-philippines/v1/9.svg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59c6c7fb-868e-40e6-9143-45628f98164a",
   "metadata": {},
   "source": [
    "## Example: LLMs Lack Access To Updated Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bc83246d-7ea4-4ed6-81db-996679d1c1f9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# should be able to answer this without additional context\n",
    "response = mistral_llm.complete(\n",
    "    \"What can you tell me about the Royal Bank of Canada?\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5c0339eb-14d5-4d60-9c5d-4b9432f280b1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The Royal Bank of Canada (RBC) is the largest bank in Canada by market capitalization and one of the largest banks in the world based on market capitalization. It was founded in Halifax, Nova Scotia in 1864, and its corporate headquarters are now located in Toronto, Ontario.\n",
      "\n",
      "RBC provides a broad range of financial services, including personal and commercial banking, wealth management, insurance, investor services, and capital markets products and services on a global basis. It serves more than 16 million clients and has over 80,000 employees worldwide.\n",
      "\n",
      "The bank operates through several branches across Canada and has a presence in the United States and 34 other countries. It is one of Canada's Big Five banks, a group that includes the Bank of Montreal, Canadian Imperial Bank of Commerce, Bank of Nova Scotia, and Toronto-Dominion Bank.\n",
      "\n",
      "RBC is known for its commitment to corporate social responsibility and has been recognized for its efforts in areas such as environmental sustainability, community involvement, and diversity and inclusion.\n",
      "\n",
      "As with any financial institution, it's always a good idea to do your own research and due diligence before making any financial decisions.\n"
     ]
    }
   ],
   "source": [
    "print(response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8a1a0204-a42b-4132-8029-0cf86c0e2e90",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I don't have real-time data access to provide the exact percentage of promotions given to women employees in the 2023 Engagement Survey. Please refer to the original survey report or check with the organization that conducted the survey for the most accurate information.\n"
     ]
    }
   ],
   "source": [
    "# a query that needs Annual Engagement Survey 2023\n",
    "query = \"According to the 2023 Engagement Survey, what percentage of promotions were given to women employees?\"\n",
    "\n",
    "response = mistral_llm.complete(query)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "48fb1cb1-b078-4004-9ec9-035a354b37ef",
   "metadata": {},
   "source": [
    "![Divider Image](https://d3ddy8balm3goa.cloudfront.net/mlops-rag-workshop/divider-2.excalidraw.svg)\n",
    "![Slide Ten](https://d3ddy8balm3goa.cloudfront.net/genai-philippines/v1/10.svg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "38d24db0-d19e-47ff-8282-3a03f798604f",
   "metadata": {},
   "source": [
    "## Example: RAG Yields More Accurate Responses"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8cda0969-7314-4eef-8ddd-db0ae80ae219",
   "metadata": {},
   "outputs": [],
   "source": [
    "!mkdir data\n",
    "!wget \"https://www.rbc.com/investor-relations/_assets-custom/pdf/ar_2023_e.pdf\" -O \"./data/RBC-Annual-Report-2023.pdf\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8a773f87-64c5-40da-bb3e-3db96d4d6230",
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index.core import SimpleDirectoryReader, VectorStoreIndex\n",
    "\n",
    "# build an in-memory RAG over the Annual Report in 4 lines of code\n",
    "loader = SimpleDirectoryReader(input_dir=\"./data\")\n",
    "documents = loader.load_data()\n",
    "index = VectorStoreIndex.from_documents(documents)\n",
    "rag = index.as_query_engine(llm=mistral_llm)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "207e8d84-d6d5-45b7-9f8c-b21ba8658a2e",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = rag.query(query)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d76260a1-ddaa-48ee-acc6-6b99a6e0d5f6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "According to the data provided, 54% of promotions were given to women employees. This is based on self-identification and excludes certain groups such as summer interns, students, co-ops, City National Bank and RBC Brewin Dolphin. The information is part of the company's focus on Diversity & Inclusion.\n"
     ]
    }
   ],
   "source": [
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "06f796b1-cd3f-41a0-81e3-5326bf409889",
   "metadata": {},
   "source": [
    "![Divider Image](https://d3ddy8balm3goa.cloudfront.net/mlops-rag-workshop/divider-2.excalidraw.svg)\n",
    "![Slide Eleven](https://d3ddy8balm3goa.cloudfront.net/genai-philippines/v1/11.svg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab094120-9669-41dc-adb0-a10343e3acef",
   "metadata": {},
   "source": [
    "## Example: 3 Steps For Basic RAG (Unpacking the previous Example RAG)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2c02b0a2-dfe4-495c-8810-21998d35b280",
   "metadata": {},
   "source": [
    "### Step 1: Build Knowledge Store"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f108a3d4-4e2a-4e3d-a20b-1f3e122cfd78",
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"Load the data.\n",
    "\n",
    "With llama-index, before any transformations are applied,\n",
    "data is loaded in the `Document` abstraction, which is\n",
    "a container that holds the text of the document.\n",
    "\"\"\"\n",
    "\n",
    "from llama_index.core import SimpleDirectoryReader\n",
    "\n",
    "loader = SimpleDirectoryReader(input_dir=\"./data\")\n",
    "documents = loader.load_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3dba1e90-4cd7-4394-a8ba-e7ed52c35729",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'2  |  Royal Bank of Canada Annual Report 2023Our Purpose\\nHelping clients thrive and  \\ncommunities prosper\\nGuided by our Vision  to be among the world’s most trusted \\nand successful financial institutions, and driven by our \\nPurpose, we aim to be:\\nIn Canada:  \\nthe undisputed leader in financial services\\nIn the United States:  \\nthe preferred partner to corporate, institutional and high  \\nnet worth clients and their businesses\\nIn select global financial centres:   \\na leading financial services partner valued for our expertise\\nConnect with us\\n  facebook.com/rbc\\n  instagram.com/rbc x.com/rbc\\n  youtube.com/user/RBC  linkedin.com/company/rbc\\n tiktok.com/@rbcFor more information on how we \\nare leading with Purpose in creating \\ndifferentiated value for our clients, \\ncommunities, employees and \\nshareholders, please visit  \\nRBC Stories .We are guided by our Values :\\n\\uf0a1   Client First\\n\\uf0a1    Collaboration\\n\\uf0a1   Accountability\\n\\uf0a1   Diversity & Inclusion\\n\\uf0a1   Integri ty'"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# if you want to see what the text looks like\n",
    "documents[1].text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4a2c764c-ef6b-41ce-9ba4-c8a54ac75990",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:Payload indexes have no effect in the local Qdrant. Please use server Qdrant if you need payload indexes.\n"
     ]
    }
   ],
   "source": [
    "\"\"\"Chunk, Encode, and Store into a Vector Store.\n",
    "\n",
    "To streamline the process, we can make use of the IngestionPipeline\n",
    "class that will apply your specified transformations to the\n",
    "Document's.\n",
    "\"\"\"\n",
    "\n",
    "from llama_index.core.ingestion import IngestionPipeline\n",
    "from llama_index.core.node_parser import SentenceSplitter\n",
    "from llama_index.embeddings.openai import OpenAIEmbedding\n",
    "from llama_index.vector_stores.qdrant import QdrantVectorStore\n",
    "import qdrant_client\n",
    "\n",
    "client = qdrant_client.QdrantClient(location=\":memory:\")\n",
    "vector_store = QdrantVectorStore(client=client, collection_name=\"test_store\")\n",
    "\n",
    "pipeline = IngestionPipeline(\n",
    "    transformations=[\n",
    "        SentenceSplitter(),\n",
    "        OpenAIEmbedding(),\n",
    "    ],\n",
    "    vector_store=vector_store,\n",
    ")\n",
    "_nodes = pipeline.run(documents=documents, num_workers=4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "53303b32-eb64-40e5-9589-c9cb81378631",
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"Create a llama-index... wait for it... Index.\n",
    "\n",
    "After uploading your encoded documents into your vector\n",
    "store of choice, you can connect to it with a VectorStoreIndex\n",
    "which then gives you access to all of the llama-index functionality.\n",
    "\"\"\"\n",
    "\n",
    "from llama_index.core import VectorStoreIndex\n",
    "\n",
    "index = VectorStoreIndex.from_vector_store(vector_store=vector_store)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db2c0528-875a-4bf5-9372-358af9ae3d4e",
   "metadata": {},
   "source": [
    "### Step 2: Retrieve Against A Query"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "48d6c333-06fb-4bc5-ba86-f4712d25b7fa",
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"Retrieve relevant documents against a query.\n",
    "\n",
    "With our Index ready, we can now query it to\n",
    "retrieve the most relevant document chunks.\n",
    "\"\"\"\n",
    "\n",
    "retriever = index.as_retriever(similarity_top_k=2)\n",
    "retrieved_nodes = retriever.retrieve(query)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7ab5eacd-0738-4f1f-94b7-af7872c8cd3f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[NodeWithScore(node=TextNode(id_='48cf88c2-cca9-435b-bd59-1f7cdfaa6251', embedding=None, metadata={'page_label': '15', 'file_name': 'RBC-Annual-Report-2023.pdf', 'file_path': '/Users/nerdai/talks/2024/genai-philippines/data/RBC-Annual-Report-2023.pdf', 'file_type': 'application/pdf', 'file_size': 7571657, 'creation_date': '2024-06-22', 'last_modified_date': '2023-11-29'}, excluded_embed_metadata_keys=['file_name', 'file_type', 'file_size', 'creation_date', 'last_modified_date', 'last_accessed_date'], excluded_llm_metadata_keys=['file_name', 'file_type', 'file_size', 'creation_date', 'last_modified_date', 'last_accessed_date'], relationships={<NodeRelationship.SOURCE: '1'>: RelatedNodeInfo(node_id='f84f0ceb-9036-4b26-82b9-f2cbcd8b039f', node_type=<ObjectType.DOCUMENT: '4'>, metadata={'page_label': '15', 'file_name': 'RBC-Annual-Report-2023.pdf', 'file_path': '/Users/nerdai/talks/2024/genai-philippines/data/RBC-Annual-Report-2023.pdf', 'file_type': 'application/pdf', 'file_size': 7571657, 'creation_date': '2024-06-22', 'last_modified_date': '2023-11-29'}, hash='0be66564624ef899bc5dd8a85dcc835d36489339753c2c415191fb0a93d58f2d')}, text='Royal Bank of Canada Annual Report 2023  |  15\\nWomen  represented: \\n\\uf0a1  49% of hires(5)\\n\\uf0a1  54% of promotions(6)\\n\\uf0a1   43% of new executive \\nappointments(7) , relative to  \\nour goal of 50%Black, Indigenous and People  \\nof Colour (BIPOC)  represented:\\n\\uf0a1  61% of hires(5)\\n\\uf0a1  45% of promotions(6)\\n\\uf0a1   25% of new executive \\nappointments(7) , relative to our \\ngoal of 30%Continued our focus on Diversity & Inclusion\\n(1) MediaCorp Canada Inc.\\n(2) Great Place to Work Institute\\n(3) Diversity Champion Talent Award for companies above 10,000 employees, LinkedIn\\n(4)  Learning hours encompass the cumulative time devoted to various learning initiatives during fiscal 2023\\n(5)  Hires includes new external hires and rehires globally excluding City National Bank and RBC Brewin Dolphin; based on self-identification; excludes summer interns, students and co-ops \\n(6)  Promotions are defined as an upward change in Global Grade. Excludes summer interns, students, co-ops, City National Bank and RBC Brewin Dolphin. Values represent data from our \\nglobal operations. Based on self-identification\\n(7)  A new executive appointment is the appointment of an internal employee or external hire as a first-time Vice President, Senior Vice President or Executive Vice President. Based on self-\\nidentification. Per RBC’s Diversity & Inclusion Roadmap 2025, our goal is to achieve representation of 30% BIPOC executives and 50% women executives by 2025\\n(8) Headcount under 30 globally, including City National Bank, BlueBay Asset Management and RBC Brewin Dolphin employees\\n(9) Based on self-identification\\n(10) A group that is historically underrepresented may include those who self-identify as women; Black, Indigenous, and People of Colour (BIPOC); LGBTQ+ and/or persons with disabilities\\n(11) Employee Engagement Survey conducted between April 26-May 10, 2023; participation rate was 74%Global employee base \\ncomprised of 18%  \\nyoung people(8)\\nWelcomed 1,900+  \\nsummer students  across the \\nglobe, 59% were BIPOC(9)\\nWe continued to intentionally enhance programs for historically underrepresented groups(10) to drive more \\nequitable opportunities for promotion and development.', mimetype='text/plain', start_char_idx=0, end_char_idx=2156, text_template='{metadata_str}\\n\\n{content}', metadata_template='{key}: {value}', metadata_seperator='\\n'), score=0.8332170908420787),\n",
       " NodeWithScore(node=TextNode(id_='db753cc2-51c4-4734-b90d-f67e8479f8b9', embedding=None, metadata={'page_label': '14', 'file_name': 'RBC-Annual-Report-2023.pdf', 'file_path': '/Users/nerdai/talks/2024/genai-philippines/data/RBC-Annual-Report-2023.pdf', 'file_type': 'application/pdf', 'file_size': 7571657, 'creation_date': '2024-06-22', 'last_modified_date': '2023-11-29'}, excluded_embed_metadata_keys=['file_name', 'file_type', 'file_size', 'creation_date', 'last_modified_date', 'last_accessed_date'], excluded_llm_metadata_keys=['file_name', 'file_type', 'file_size', 'creation_date', 'last_modified_date', 'last_accessed_date'], relationships={<NodeRelationship.SOURCE: '1'>: RelatedNodeInfo(node_id='33047b38-f645-498a-9ffe-85c7a141838c', node_type=<ObjectType.DOCUMENT: '4'>, metadata={'page_label': '14', 'file_name': 'RBC-Annual-Report-2023.pdf', 'file_path': '/Users/nerdai/talks/2024/genai-philippines/data/RBC-Annual-Report-2023.pdf', 'file_type': 'application/pdf', 'file_size': 7571657, 'creation_date': '2024-06-22', 'last_modified_date': '2023-11-29'}, hash='98da66fe2b97c52d13ee8ed13b4ec8b24392aecc7ce641335cfe87b5f4eb7637')}, text='14  |  Royal Bank of Canada Annual Report 2023\\nEmployees\\nOur strength and success is rooted in the 94,000+ employees worldwide who \\nlive our Purpose and Values. RBC is committed to an inclusive workplace culture \\nthat engages, supports and empowers our employees to help clients thrive and \\ncommunities prosper.\\nAmong Canada’s Top 100 Employers  and Best \\nWorkplaces  in 2023(1)(2)\\nOne of Canada’s Best Diversity Employers(2) and a \\nDiversity Champion Talent Award  recipient(3)\\nOngoing learning is a cornerstone of how we \\nsupport our colleagues’ professional development \\nand career aspirations. Our global workforce \\ncollectively invested 3 million hours  in building their \\ntechnical and business skills(4)\\nApproximately $19+ billion  in competitive \\ncompensation and benefits, including increased \\nemployee matching to our defined-contribution \\npension plan\\nRecognized as one of Canada’s Top Employers for \\nYoung People(1)2023 highlights across our balanced scorecard\\n93%\\nfeel they contribute to  \\nRBC’s success2023 Employee Engagement Survey  found employees are highly engaged and feel \\nproud to be part of RBC(11) \\n88%\\nare proud to be part  \\nof RBC87%\\nare willing to go above  \\nand beyond', mimetype='text/plain', start_char_idx=0, end_char_idx=1196, text_template='{metadata_str}\\n\\n{content}', metadata_template='{key}: {value}', metadata_seperator='\\n'), score=0.798579112930379)]"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# to view the retrieved nodes\n",
    "retrieved_nodes"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "682d1e8c-a579-4449-b60d-8f2bd98ad1ee",
   "metadata": {},
   "source": [
    "### Step 3: Generate Final Response"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "08e65509-ab2c-49bd-9dc1-af38e389b40c",
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"Context-Augemented Generation.\n",
    "\n",
    "With our Index ready, we can create a QueryEngine\n",
    "that handles the retrieval and context augmentation\n",
    "in order to get the final response.\n",
    "\"\"\"\n",
    "\n",
    "query_engine = index.as_query_engine(llm=mistral_llm)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "31d94f08-db5b-452b-bb24-cf087d6c5307",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Context information is below.\n",
      "---------------------\n",
      "{context_str}\n",
      "---------------------\n",
      "Given the context information and not prior knowledge, answer the query.\n",
      "Query: {query_str}\n",
      "Answer: \n"
     ]
    }
   ],
   "source": [
    "# to inspect the default prompt being used\n",
    "print(\n",
    "    query_engine.get_prompts()[\n",
    "        \"response_synthesizer:text_qa_template\"\n",
    "    ].default_template.template\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1439a11f-cba5-4b80-8c4c-edbd43026b93",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "According to the data provided, 54% of promotions were given to women employees. This is part of the organization's ongoing commitment to diversity and inclusion.\n"
     ]
    }
   ],
   "source": [
    "response = query_engine.query(query)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "06113973-f7f5-42fe-96bf-2a14be1b6c84",
   "metadata": {},
   "source": [
    "![Divider Image](https://d3ddy8balm3goa.cloudfront.net/mlops-rag-workshop/divider-2.excalidraw.svg)\n",
    "![Slide Twelve](https://d3ddy8balm3goa.cloudfront.net/genai-philippines/v1/12.svg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "821424ae-8567-4919-95e1-385c4cd07052",
   "metadata": {},
   "source": [
    "[Hi-Resolution Cheat Sheet](https://d3ddy8balm3goa.cloudfront.net/llamaindex/rag-cheat-sheet-final.svg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a990db36-125c-461a-a4c8-bb50c0cf5f7f",
   "metadata": {},
   "source": [
    "## Example: Graph RAG"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c3f71807-8fc4-4d29-b603-4db763daa741",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/nerdai/.pyenv/versions/3.11.3/envs/genai-philippines/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n",
      "Parsing nodes: 100%|█████████████████████████████████████████████████████████████████████| 10/10 [00:00<00:00, 300.08it/s]\n",
      "Extracting paths from text: 100%|█████████████████████████████████████████████████████████| 10/10 [00:06<00:00,  1.43it/s]\n",
      "Extracting implicit paths: 100%|███████████████████████████████████████████████████████| 10/10 [00:00<00:00, 35848.75it/s]\n",
      "Generating embeddings: 100%|████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  2.01it/s]\n",
      "Generating embeddings: 100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00,  3.44it/s]\n"
     ]
    }
   ],
   "source": [
    "from llama_index.core import PropertyGraphIndex\n",
    "from llama_index.embeddings.openai import OpenAIEmbedding\n",
    "\n",
    "index = PropertyGraphIndex.from_documents(\n",
    "    documents[10:20],\n",
    "    llm=openai_llm,\n",
    "    embed_model=OpenAIEmbedding(model_name=\"text-embedding-ada-002\"),\n",
    "    show_progress=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "04c8105a-096f-4b6d-8a7b-47c3703a2f9b",
   "metadata": {},
   "outputs": [],
   "source": [
    "index.property_graph_store.save_networkx_graph(name=\"./kg.html\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bd5b7f9f-de05-4b86-9b03-3e37ce4c2c70",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Bipoc -> Represented -> 45% of promotions\n",
      "Women -> Represented -> 54% of promotions\n",
      "Women -> Represented -> 43% of new executive appointments\n",
      "Women -> Represented -> 49% of hires\n"
     ]
    }
   ],
   "source": [
    "retriever = index.as_retriever(\n",
    "    include_text=False,  # include source text, default True\n",
    ")\n",
    "\n",
    "nodes = retriever.retrieve(query)\n",
    "\n",
    "for node in nodes:\n",
    "    print(node.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "606dce46-5e85-4ef0-bd1d-dddd16789e4e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "According to the 2023 Engagement Survey, 54% of promotions were given to women employees.\n"
     ]
    }
   ],
   "source": [
    "query_engine = index.as_query_engine(\n",
    "    include_text=True,\n",
    ")\n",
    "\n",
    "response = query_engine.query(query)\n",
    "\n",
    "print(str(response))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a3bcd491-b438-48e8-ade3-f5219fe1c308",
   "metadata": {},
   "source": [
    "![Divider Image](https://d3ddy8balm3goa.cloudfront.net/mlops-rag-workshop/divider-2.excalidraw.svg)\n",
    "![Slide Thirteen](https://d3ddy8balm3goa.cloudfront.net/genai-philippines/v1/13.svg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7f024428-379c-4c0d-bbdd-5909e569a5b2",
   "metadata": {},
   "source": [
    "![Divider Image](https://d3ddy8balm3goa.cloudfront.net/mlops-rag-workshop/divider-2.excalidraw.svg)\n",
    "![Slide Fourteen](https://d3ddy8balm3goa.cloudfront.net/genai-philippines/v1/14.svg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "716216ec-3d15-405b-ac47-67d454618516",
   "metadata": {},
   "source": [
    "![Divider Image](https://d3ddy8balm3goa.cloudfront.net/mlops-rag-workshop/divider-2.excalidraw.svg)\n",
    "![Slide Fifteen](https://d3ddy8balm3goa.cloudfront.net/genai-philippines/v1/15.svg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "24fdb303-d471-42c4-8eb6-2e82dc631fc5",
   "metadata": {},
   "source": [
    "## Example: Agent Ingredients — Tool Use\n",
    "\n",
    "**Note:** LLMs are not very good pseudo-random number generators (see my [LinkedIn post](https://www.linkedin.com/posts/nerdai_heres-s-fun-mini-experiment-the-activity-7193715824493219841-6AWt?utm_source=share&utm_medium=member_desktop) about this)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9f9ac3f5-610a-4006-ad8c-41269230e680",
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index.core.tools import FunctionTool\n",
    "from llama_index.agent.openai import OpenAIAgent\n",
    "from numpy import random\n",
    "from typing import List"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9344061a-1609-4b8d-ba05-3aa115f2ad0a",
   "metadata": {},
   "outputs": [],
   "source": [
    "def uniform_random_sample(n: int) -> List[float]:\n",
    "    \"\"\"Generate a list a of uniform random numbers of size n between 0 and 1.\"\"\"\n",
    "    return random.rand(n).tolist()\n",
    "\n",
    "\n",
    "rs_tool = FunctionTool.from_defaults(fn=uniform_random_sample)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8ec77eb9-270f-4c4e-badd-ee8378f1d72b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Added user message to memory: Can you please give me a sample of 10 uniformly random numbers?\n",
      "=== Calling Function ===\n",
      "Calling function: uniform_random_sample with args: {\"n\":10}\n",
      "Got output: [0.8995277304883874, 0.9085039986112343, 0.8709820611695793, 0.7730695747942606, 0.5212311221928344, 0.9430700977083472, 0.4243954511057493, 0.024527217425000303, 0.7473927621289647, 0.13689797800590942]\n",
      "========================\n",
      "\n",
      "Here is a sample of 10 uniformly random numbers:\n",
      "\n",
      "1. 0.8995277304883874\n",
      "2. 0.9085039986112343\n",
      "3. 0.8709820611695793\n",
      "4. 0.7730695747942606\n",
      "5. 0.5212311221928344\n",
      "6. 0.9430700977083472\n",
      "7. 0.4243954511057493\n",
      "8. 0.024527217425000303\n",
      "9. 0.7473927621289647\n",
      "10. 0.13689797800590942\n"
     ]
    }
   ],
   "source": [
    "agent = OpenAIAgent.from_tools([rs_tool], llm=openai_llm, verbose=True)\n",
    "\n",
    "response = agent.chat(\n",
    "    \"Can you please give me a sample of 10 uniformly random numbers?\"\n",
    ")\n",
    "print(str(response))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca6f2caa-eb71-4248-997c-7cf6906ff03f",
   "metadata": {},
   "source": [
    "![Divider Image](https://d3ddy8balm3goa.cloudfront.net/mlops-rag-workshop/divider-2.excalidraw.svg)\n",
    "![Slide Sixteen](https://d3ddy8balm3goa.cloudfront.net/genai-philippines/v1/16.svg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "122955d6-9ea7-455e-853c-977005f0f003",
   "metadata": {},
   "source": [
    "## Example: Agent Ingredients — Composable Memory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0e607609-93b2-499d-bfee-62e9dcae7db8",
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index.core.memory import (\n",
    "    VectorMemory,\n",
    "    SimpleComposableMemory,\n",
    "    ChatMemoryBuffer,\n",
    ")\n",
    "from llama_index.core.agent import FunctionCallingAgentWorker"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "61538923-8796-482f-b7a8-8b0768c7d762",
   "metadata": {},
   "outputs": [],
   "source": [
    "vector_memory = VectorMemory.from_defaults(\n",
    "    vector_store=None,  # leave as None to use default in-memory vector store\n",
    "    embed_model=OpenAIEmbedding(),\n",
    "    retriever_kwargs={\"similarity_top_k\": 2},\n",
    ")\n",
    "\n",
    "chat_memory_buffer = ChatMemoryBuffer.from_defaults()\n",
    "\n",
    "composable_memory = SimpleComposableMemory.from_defaults(\n",
    "    primary_memory=chat_memory_buffer,\n",
    "    secondary_memory_sources=[vector_memory],\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "70bbf4c6-9ef4-4eb8-be72-08d52249142f",
   "metadata": {},
   "outputs": [],
   "source": [
    "def multiply(a: int, b: int) -> int:\n",
    "    \"\"\"Multiply two integers and returns the result integer.\"\"\"\n",
    "    return a * b\n",
    "\n",
    "\n",
    "def mystery(a: int, b: int) -> int:\n",
    "    \"\"\"Mystery function on two numbers.\"\"\"\n",
    "    return a**2 - b**2\n",
    "\n",
    "\n",
    "multiply_tool = FunctionTool.from_defaults(fn=multiply)\n",
    "mystery_tool = FunctionTool.from_defaults(fn=mystery)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3f8d9a6b-b67e-4f71-b92e-b8f0f968ad47",
   "metadata": {},
   "outputs": [],
   "source": [
    "agent_worker = FunctionCallingAgentWorker.from_tools(\n",
    "    [multiply_tool, mystery_tool], llm=openai_llm, verbose=True\n",
    ")\n",
    "agent = agent_worker.as_agent(memory=composable_memory)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d32e9a73-0d6d-451f-9796-f99e7dbb26be",
   "metadata": {},
   "source": [
    "### Execute some function calls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "34b83e55-6f48-425a-b5c9-5349fe89b160",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Added user message to memory: What is the mystery function on 5 and 6?\n",
      "=== Calling Function ===\n",
      "Calling function: mystery with args: {\"a\": 5, \"b\": 6}\n",
      "=== Function Output ===\n",
      "-11\n",
      "=== LLM Response ===\n",
      "The result of the mystery function on 5 and 6 is -11.\n"
     ]
    }
   ],
   "source": [
    "response = agent.chat(\"What is the mystery function on 5 and 6?\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "00f3be76-3b1b-41dc-abc0-9c6213445a0f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Added user message to memory: What happens if you multiply 2 and 3?\n",
      "=== Calling Function ===\n",
      "Calling function: multiply with args: {\"a\": 2, \"b\": 3}\n",
      "=== Function Output ===\n",
      "6\n",
      "=== LLM Response ===\n",
      "Multiplying 2 and 3 gives you 6.\n"
     ]
    }
   ],
   "source": [
    "response = agent.chat(\"What happens if you multiply 2 and 3?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b2e22f8-f59c-4449-b39a-dcba8049a609",
   "metadata": {},
   "source": [
    "### New Agent Session"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "74cff862-ad5a-40fd-a199-99b2a61fbcf3",
   "metadata": {},
   "source": [
    "#### Without memory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8d029d3b-97a5-44db-9355-d937143b40bf",
   "metadata": {},
   "outputs": [],
   "source": [
    "agent_worker = FunctionCallingAgentWorker.from_tools(\n",
    "    [multiply_tool, mystery_tool], llm=openai_llm, verbose=True\n",
    ")\n",
    "agent_without_memory = agent_worker.as_agent()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8eabb72f-3681-42e3-ad1c-55c02a2bc950",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Added user message to memory: What was the output of the mystery function on 5 and 6 again? Don't recompute.\n",
      "=== LLM Response ===\n",
      "I don't have the ability to recall past interactions or outputs. If you need the result of the mystery function on 5 and 6, I can recompute it for you. Would you like me to do that?\n"
     ]
    }
   ],
   "source": [
    "response = agent_without_memory.chat(\n",
    "    \"What was the output of the mystery function on 5 and 6 again? Don't recompute.\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4e40632a-47c2-46c4-8df8-818f7eb36aef",
   "metadata": {},
   "source": [
    "#### With memory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6843bf31-8930-40aa-9df2-6ed2434b71f1",
   "metadata": {},
   "outputs": [],
   "source": [
    "llm = OpenAI(model=\"gpt-3.5-turbo-0613\")\n",
    "agent_worker = FunctionCallingAgentWorker.from_tools(\n",
    "    [multiply_tool, mystery_tool], llm=openai_llm, verbose=True\n",
    ")\n",
    "composable_memory = SimpleComposableMemory.from_defaults(\n",
    "    primary_memory=ChatMemoryBuffer.from_defaults(),\n",
    "    secondary_memory_sources=[\n",
    "        vector_memory.copy(\n",
    "            deep=True\n",
    "        )  # using a copy here for illustration purposes\n",
    "        # later will use original vector_memory again\n",
    "    ],\n",
    ")\n",
    "agent_with_memory = agent_worker.as_agent(memory=composable_memory)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7c231bbd-c629-4376-a97b-104ef867536e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "agent_with_memory.chat_history  # an empty chat history"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "33e98b79-fb92-4b9c-872f-3d94d589f883",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Added user message to memory: What was the output of the mystery function on 5 and 6 again? Don't recompute.\n",
      "=== LLM Response ===\n",
      "The output of the mystery function on 5 and 6 was -11.\n"
     ]
    }
   ],
   "source": [
    "response = agent_with_memory.chat(\n",
    "    \"What was the output of the mystery function on 5 and 6 again? Don't recompute.\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "001c8637-3561-4dd0-a128-9573986d6a3c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Added user message to memory: What was the output of the multiply function on 2 and 3 again? Don't recompute.\n",
      "=== LLM Response ===\n",
      "The output of the multiply function on 2 and 3 was 6.\n"
     ]
    }
   ],
   "source": [
    "response = agent_with_memory.chat(\n",
    "    \"What was the output of the multiply function on 2 and 3 again? Don't recompute.\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27591b78-bc00-48b6-9e8a-8ce691fa9cb1",
   "metadata": {},
   "source": [
    "#### Under the hood\n",
    "\n",
    "Calling `.chat()` will invoke `memory.get()`. For `SimpleComposableMemory` memory retrieved from secondary sources get added to the system prompt of the main memory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a3951a2e-8e6c-4cc9-a328-41cbfe013586",
   "metadata": {},
   "outputs": [],
   "source": [
    "composable_memory = SimpleComposableMemory.from_defaults(\n",
    "    primary_memory=ChatMemoryBuffer.from_defaults(),\n",
    "    secondary_memory_sources=[\n",
    "        vector_memory.copy(\n",
    "            deep=True\n",
    "        )  # copy for illustrative purposes to explain what\n",
    "        # happened under the hood from previous subsection\n",
    "    ],\n",
    ")\n",
    "agent_with_memory = agent_worker.as_agent(memory=composable_memory)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0072e2b8-8c46-4258-828f-24ce9ceadd65",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "system: You are a helpful assistant.\n",
      "\n",
      "Below are a set of relevant dialogues retrieved from potentially several memory sources:\n",
      "\n",
      "=====Relevant messages from memory source 1=====\n",
      "\n",
      "\tUSER: What is the mystery function on 5 and 6?\n",
      "\tASSISTANT: None\n",
      "\tTOOL: -11\n",
      "\tASSISTANT: The result of the mystery function on 5 and 6 is -11.\n",
      "\n",
      "=====End of relevant messages from memory source 1======\n",
      "\n",
      "This is the end of the retrieved message dialogues.\n"
     ]
    }
   ],
   "source": [
    "print(\n",
    "    agent_with_memory.memory.get(\n",
    "        \"What was the output of the mystery function on 5 and 6 again? Don't recompute.\"\n",
    "    )[0]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "58ec1f66-4e81-4092-adef-e0c25a52feec",
   "metadata": {},
   "source": [
    "![Divider Image](https://d3ddy8balm3goa.cloudfront.net/mlops-rag-workshop/divider-2.excalidraw.svg)\n",
    "![Slide Seventeen](https://d3ddy8balm3goa.cloudfront.net/genai-philippines/v1/17.svg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "134354bf-6901-481a-87da-d4a41cb6cd79",
   "metadata": {},
   "source": [
    "## Example: Reflection Toxicity Reduction\n",
    "\n",
    "Here, we'll use llama-index `TollInteractiveReflectionAgent` to perform reflection and correction cycles on potentially harmful text. See the full demo [here](https://github.com/run-llama/llama_index/blob/main/docs/examples/agent/introspective_agent_toxicity_reduction.ipynb).\n",
    "\n",
    "The first thing we will do here is define the `PerspectiveTool`, which our `ToolInteractiveReflectionAgent` will make use of thru another agent, namely a `CritiqueAgent`.\n",
    "\n",
    "To use Perspecive's API, you will need to do the following steps:\n",
    "\n",
    "1. Enable the Perspective API in your Google Cloud projects\n",
    "2. Generate a new set of credentials (i.e. API key) that you will need to either set an env var `PERSPECTIVE_API_KEY` or supply directly in the appropriate parts of the code that follows.\n",
    "\n",
    "To perform steps 1. and 2., you can follow the instructions outlined here: https://developers.perspectiveapi.com/s/docs-enable-the-api?language=en_US."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "64e8c235-4fef-450b-9e60-a879455da6af",
   "metadata": {},
   "source": [
    "### Perspective API as Tool"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4fcd3dd0-226c-4f34-b8cd-eab20e4a9363",
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index.core.bridge.pydantic import Field\n",
    "\n",
    "from googleapiclient import discovery\n",
    "from typing import Dict, Optional, Tuple\n",
    "import os"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9bd14695-5cff-4000-a104-981275c854f9",
   "metadata": {},
   "outputs": [],
   "source": [
    "class Perspective:\n",
    "    \"\"\"Custom class to interact with Perspective API.\"\"\"\n",
    "\n",
    "    attributes = [\n",
    "        \"toxicity\",\n",
    "        \"severe_toxicity\",\n",
    "        \"identity_attack\",\n",
    "        \"insult\",\n",
    "        \"profanity\",\n",
    "        \"threat\",\n",
    "        \"sexually_explicit\",\n",
    "    ]\n",
    "\n",
    "    def __init__(self, api_key: Optional[str] = None) -> None:\n",
    "        if api_key is None:\n",
    "            try:\n",
    "                api_key = os.environ[\"PERSPECTIVE_API_KEY\"]\n",
    "            except KeyError:\n",
    "                raise ValueError(\n",
    "                    \"Please provide an api key or set PERSPECTIVE_API_KEY env var.\"\n",
    "                )\n",
    "\n",
    "        self._client = discovery.build(\n",
    "            \"commentanalyzer\",\n",
    "            \"v1alpha1\",\n",
    "            developerKey=api_key,\n",
    "            discoveryServiceUrl=\"https://commentanalyzer.googleapis.com/$discovery/rest?version=v1alpha1\",\n",
    "            static_discovery=False,\n",
    "        )\n",
    "\n",
    "    def get_toxicity_scores(self, text: str) -> Dict[str, float]:\n",
    "        \"\"\"Function that makes API call to Perspective to get toxicity scores across various attributes.\"\"\"\n",
    "        analyze_request = {\n",
    "            \"comment\": {\"text\": text},\n",
    "            \"requestedAttributes\": {\n",
    "                att.upper(): {} for att in self.attributes\n",
    "            },\n",
    "        }\n",
    "\n",
    "        response = (\n",
    "            self._client.comments().analyze(body=analyze_request).execute()\n",
    "        )\n",
    "        try:\n",
    "            return {\n",
    "                att: response[\"attributeScores\"][att.upper()][\"summaryScore\"][\n",
    "                    \"value\"\n",
    "                ]\n",
    "                for att in self.attributes\n",
    "            }\n",
    "        except Exception as e:\n",
    "            raise ValueError(\"Unable to parse response\") from e\n",
    "\n",
    "\n",
    "perspective = Perspective()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ac230f97-3562-4df3-b611-f529085c7287",
   "metadata": {},
   "outputs": [],
   "source": [
    "def perspective_function_tool(\n",
    "    text: str = Field(\n",
    "        default_factory=str,\n",
    "        description=\"The text to compute toxicity scores on.\",\n",
    "    ),\n",
    ") -> Tuple[str, float]:\n",
    "    \"\"\"Returns the toxicity score of the most problematic toxic attribute.\"\"\"\n",
    "    scores = perspective.get_toxicity_scores(text=text)\n",
    "    max_key = max(scores, key=scores.get)\n",
    "    return (max_key, scores[max_key] * 100)\n",
    "\n",
    "\n",
    "from llama_index.core.tools import FunctionTool\n",
    "\n",
    "pespective_tool = FunctionTool.from_defaults(\n",
    "    perspective_function_tool,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "121b73e7-d45c-4fd4-9f1a-33b5c055fd6a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('toxicity', 2.6028076)"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "perspective_function_tool(text=\"friendly greetings from python\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad35861b-a162-4590-a2ba-53f917d53b78",
   "metadata": {},
   "source": [
    "### Build Agent To Reduce Toxicity of Harmful Text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b510f01c-6619-417b-b040-ac2a7e744431",
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index.agent.introspective import IntrospectiveAgentWorker\n",
    "from llama_index.agent.introspective import (\n",
    "    ToolInteractiveReflectionAgentWorker,\n",
    ")\n",
    "from llama_index.core.llms import ChatMessage, MessageRole"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8518fc2c-36c2-4144-b3ad-3c206be99c28",
   "metadata": {},
   "outputs": [],
   "source": [
    "verbose = True\n",
    "critique_agent_worker = FunctionCallingAgentWorker.from_tools(\n",
    "    tools=[pespective_tool], llm=OpenAI(\"gpt-3.5-turbo\"), verbose=verbose\n",
    ")\n",
    "correction_llm = OpenAI(\"gpt-4-turbo-preview\")\n",
    "\n",
    "\n",
    "def stopping_callable(critique_str: str) -> bool:\n",
    "    \"\"\"\n",
    "    Function that determines stopping condition for reflection & correction cycles.\n",
    "\n",
    "    critique_str [str]: The response string provided by the critique agent.\n",
    "    \"\"\"\n",
    "    return \"[PASS]\" in critique_str\n",
    "\n",
    "\n",
    "tool_interactive_reflection_agent_worker = (\n",
    "    ToolInteractiveReflectionAgentWorker.from_defaults(\n",
    "        critique_agent_worker=critique_agent_worker,\n",
    "        critique_template=(\n",
    "            \"Please provide a toxicity critique for the provided text. \"\n",
    "            \"If the toxicity score is less than 3.0 then please \"\n",
    "            \"write '[PASS]' otherwise write '[FAIL]'. \"\n",
    "            \"Here is the text:\\n {input_str}\"\n",
    "        ),\n",
    "        stopping_callable=stopping_callable,\n",
    "        correction_llm=correction_llm,\n",
    "        verbose=verbose,\n",
    "    )\n",
    ")\n",
    "\n",
    "introspective_agent_worker = IntrospectiveAgentWorker.from_defaults(\n",
    "    reflective_agent_worker=tool_interactive_reflection_agent_worker,\n",
    "    verbose=verbose,\n",
    ")\n",
    "\n",
    "chat_history = [\n",
    "    ChatMessage(\n",
    "        content=\"You are an assistant that generates safer versions of potentially toxic, user-supplied text.\",\n",
    "        role=MessageRole.SYSTEM,\n",
    "    )\n",
    "]\n",
    "\n",
    "introspective_agent = introspective_agent_worker.as_agent(\n",
    "    chat_history=chat_history, verbose=verbose\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d2cf0716-9188-43c6-b7e2-9ef8c95db84b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "> Running step 6e267895-b083-46a9-8f0c-ef6986c819fd. Step input: Those individuals who publicly tout eating no meat, especially when their stated reason has to do with caring about animals, are some of the most hypocritical people I’ve ever met. One vegan acquaintance routinely beat her\n",
      "Added user message to memory: Those individuals who publicly tout eating no meat, especially when their stated reason has to do with caring about animals, are some of the most hypocritical people I’ve ever met. One vegan acquaintance routinely beat her\n",
      "> Running step 7c3dafd7-6c31-401a-8e1c-6f5e6ca2f18c. Step input: Please provide a toxicity critique for the provided text. If the toxicity score is less than 3.0 then please write '[PASS]' otherwise write '[FAIL]'. Here is the text:\n",
      " Those individuals who publicly tout eating no meat, especially when their stated reason has to do with caring about animals, are some of the most hypocritical people I’ve ever met. One vegan acquaintance routinely beat her\n",
      "Added user message to memory: Please provide a toxicity critique for the provided text. If the toxicity score is less than 3.0 then please write '[PASS]' otherwise write '[FAIL]'. Here is the text:\n",
      " Those individuals who publicly tout eating no meat, especially when their stated reason has to do with caring about animals, are some of the most hypocritical people I’ve ever met. One vegan acquaintance routinely beat her\n",
      "=== Calling Function ===\n",
      "Calling function: perspective_function_tool with args: {\"text\": \"Those individuals who publicly tout eating no meat, especially when their stated reason has to do with caring about animals, are some of the most hypocritical people I\\u2019ve ever met. One vegan acquaintance routinely beat her\"}\n",
      "=== Function Output ===\n",
      "('insult', 51.01222)\n",
      "> Running step 14334cb6-2395-48c2-9a75-2205a9a42886. Step input: None\n",
      "=== LLM Response ===\n",
      "[FAIL] The toxicity score for the provided text is 51.01, which is above 3.0.\n",
      "Critique: [FAIL] The toxicity score for the provided text is 51.01, which is above 3.0.\n",
      "Correction: People who choose not to eat meat for ethical reasons related to animal welfare are making a personal decision. It's important to respect diverse perspectives and experiences.\n",
      "> Running step 4b8f3d68-4931-4b7a-8ff0-b998c6ba6b39. Step input: Please provide a toxicity critique for the provided text. If the toxicity score is less than 3.0 then please write '[PASS]' otherwise write '[FAIL]'. Here is the text:\n",
      " People who choose not to eat meat for ethical reasons related to animal welfare are making a personal decision. It's important to respect diverse perspectives and experiences.\n",
      "Added user message to memory: Please provide a toxicity critique for the provided text. If the toxicity score is less than 3.0 then please write '[PASS]' otherwise write '[FAIL]'. Here is the text:\n",
      " People who choose not to eat meat for ethical reasons related to animal welfare are making a personal decision. It's important to respect diverse perspectives and experiences.\n",
      "=== Calling Function ===\n",
      "Calling function: perspective_function_tool with args: {\"text\": \"People who choose not to eat meat for ethical reasons related to animal welfare are making a personal decision. It's important to respect diverse perspectives and experiences.\"}\n",
      "=== Function Output ===\n",
      "('toxicity', 1.3634177)\n",
      "> Running step afdd2efd-6f91-4d5b-a69d-deaafca1b04a. Step input: None\n",
      "=== LLM Response ===\n",
      "[PASS] The toxicity score of the provided text is 1.36, which is less than 3.0.\n",
      "Critique: [PASS] The toxicity score of the provided text is 1.36, which is less than 3.0.\n"
     ]
    }
   ],
   "source": [
    "harmful_text = (\n",
    "    \"Those individuals who publicly tout eating no meat, \"\n",
    "    \"especially when their stated reason has to do with caring \"\n",
    "    \"about animals, are some of the most hypocritical people I’ve \"\n",
    "    \"ever met. One vegan acquaintance routinely beat her\"\n",
    ")\n",
    "response = introspective_agent.chat(harmful_text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "76fb45f1-d384-4ff7-b50d-c7d985f785d9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "People who choose not to eat meat for ethical reasons related to animal welfare are making a personal decision. It's important to respect diverse perspectives and experiences.\n"
     ]
    }
   ],
   "source": [
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "892234c8-4567-4606-bdab-fe4878b89cce",
   "metadata": {},
   "source": [
    "![Divider Image](https://d3ddy8balm3goa.cloudfront.net/mlops-rag-workshop/divider-2.excalidraw.svg)\n",
    "![Slide Eighteen](https://d3ddy8balm3goa.cloudfront.net/genai-philippines/v1/18.svg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "061cd47f-5f40-467c-8e88-edfa1474333f",
   "metadata": {},
   "source": [
    "![Divider Image](https://d3ddy8balm3goa.cloudfront.net/mlops-rag-workshop/divider-2.excalidraw.svg)\n",
    "![Slide Nineteen](https://d3ddy8balm3goa.cloudfront.net/genai-philippines/v1/19.svg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d2b1d75a-3aa1-460d-af4a-8bf79215bb2b",
   "metadata": {},
   "source": [
    "![Divider Image](https://d3ddy8balm3goa.cloudfront.net/mlops-rag-workshop/divider-2.excalidraw.svg)\n",
    "![Slide Twenty](https://d3ddy8balm3goa.cloudfront.net/genai-philippines/v1/20.svg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d94101d-e392-4268-afdb-7a5c047c6745",
   "metadata": {},
   "source": [
    "## Example: Agentic RAG"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "55954c5c-e17a-4b58-91ab-5a4f9455971c",
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index.core.tools import ToolMetadata\n",
    "from llama_index.core.tools import QueryEngineTool"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5499728a-66b3-4ea2-9bc5-026a0a5bd505",
   "metadata": {},
   "outputs": [],
   "source": [
    "!mkdir vector_data\n",
    "!wget \"https://vectorinstitute.ai/wp-content/uploads/2024/02/Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf\" -O \"./vector_data/Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "87a674de-6a14-4edc-9f58-13539488fa63",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Build basic RAG over Vector\n",
    "vector_loader = SimpleDirectoryReader(input_dir=\"./vector_data\")\n",
    "vector_documents = vector_loader.load_data()\n",
    "vector_index = VectorStoreIndex.from_documents(vector_documents)\n",
    "vector_query_engine = vector_index.as_query_engine(llm=mistral_llm)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1b94c0f5-c69f-4b9f-b4ff-e7ec3aa20811",
   "metadata": {},
   "outputs": [],
   "source": [
    "query_engine_tools = [\n",
    "    QueryEngineTool(\n",
    "        query_engine=query_engine,\n",
    "        metadata=ToolMetadata(\n",
    "            name=\"rbc_annual_report_2023\",\n",
    "            description=(\"Provides information about RBC in the year 2023.\"),\n",
    "        ),\n",
    "    ),\n",
    "    QueryEngineTool(\n",
    "        query_engine=vector_query_engine,\n",
    "        metadata=ToolMetadata(\n",
    "            name=\"vector_annual_report_2023\",\n",
    "            description=(\n",
    "                \"Provides information about Vector in the year 2023.\"\n",
    "            ),\n",
    "        ),\n",
    "    ),\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b99b898b-28a6-42d2-adf5-999cc5d47e8f",
   "metadata": {},
   "outputs": [],
   "source": [
    "agent = OpenAIAgent.from_tools(query_engine_tools, verbose=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b6591d02-5201-4a1c-8f2b-f7d580619987",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Added user message to memory: According to the 2023 Engagement Survey, what percentage of promotions were given to women employees?\n",
      "=== Calling Function ===\n",
      "Calling function: rbc_annual_report_2023 with args: {\"input\":\"percentage of promotions given to women employees\"}\n",
      "Got output: 54%\n",
      "========================\n",
      "\n"
     ]
    }
   ],
   "source": [
    "response = agent.chat(query)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "38e20c0c-1752-4443-a6ce-982287ba79ab",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "According to the 2023 Engagement Survey, 54% of promotions were given to women employees.\n"
     ]
    }
   ],
   "source": [
    "print(response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b8535c52-9b70-4873-81dd-83e97d2b28c1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Added user message to memory: According to Vector Institute's Annual Report 2022-2023, how many AI jobs were created in Ontario?\n",
      "=== Calling Function ===\n",
      "Calling function: vector_annual_report_2023 with args: {\"input\":\"number of AI jobs created in Ontario\"}\n",
      "Got output: The number of AI jobs created in Ontario is 20,634.\n",
      "========================\n",
      "\n"
     ]
    }
   ],
   "source": [
    "response = agent.chat(\n",
    "    \"According to Vector Institute's Annual Report 2022-2023, \"\n",
    "    \"how many AI jobs were created in Ontario?\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "af14c4da-aa9d-46e4-86c2-65d4d936d19c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "According to Vector Institute's Annual Report 2022-2023, 20,634 AI jobs were created in Ontario.\n"
     ]
    }
   ],
   "source": [
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db5aef92-f7c4-450c-80df-bfdcd68b6092",
   "metadata": {},
   "source": [
    "![Divider Image](https://d3ddy8balm3goa.cloudfront.net/mlops-rag-workshop/divider-2.excalidraw.svg)\n",
    "![Slide TwentyOne](https://d3ddy8balm3goa.cloudfront.net/genai-philippines/v1/21.svg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "64d852aa-230c-4f2e-80f4-a9c77a300ebd",
   "metadata": {},
   "source": [
    "## Example: Multi-hop Agent (WIP)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee0d0e06-38d7-46a9-8a18-3a92464474c1",
   "metadata": {},
   "source": [
    "At the time of this presentation, this is still ongoing work, but despite its unfinished status, it demonstrates the flexibility and advantages for using an agentic interface over extneral knowledge bases (i.e., RAG).\n",
    "\n",
    "With the multi-hop agent, we aim to solve query's by first planning out the required data elements that should be retrieved in order to be able to answer the question. And so, we're really combining here a few concepts:\n",
    "\n",
    "- planning\n",
    "- structured data extraction (using a RAG tool)\n",
    "- reflection/correction"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bc254181-0a35-4984-b2a8-c89c50ffa563",
   "metadata": {},
   "source": [
    "![multi-hop agent](https://d3ddy8balm3goa.cloudfront.net/genai-philippines/v1/multi-hop-agent.excalidraw.svg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4d446e3f-9618-47c3-a057-08c88e694866",
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, Document\n",
    "from llama_index.core.tools import QueryEngineTool\n",
    "\n",
    "index = VectorStoreIndex.from_documents([Document.example()])\n",
    "tool = QueryEngineTool.from_defaults(\n",
    "    index.as_query_engine(),\n",
    "    name=\"dummy\",\n",
    "    description=\"dummy\",\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8e078006-d2c9-490e-9881-e5cdd23a9a81",
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index.core.agent import FunctionCallingAgentWorker\n",
    "from llama_index.agent.multi_hop.planner import MultiHopPlannerAgent\n",
    "\n",
    "# create the function calling worker for reasoning\n",
    "worker = FunctionCallingAgentWorker.from_tools([tool], verbose=True)\n",
    "\n",
    "# wrap the worker in the top-level planner\n",
    "agent = MultiHopPlannerAgent(worker, tools=[tool], verbose=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cc4412ca-69d9-4324-b734-11d89adb198b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "=== Initial plan ===\n",
      "## Structured Context\n",
      "{\n",
      "    \"title\": \"StructuredContext\",\n",
      "    \"description\": \"Data class for holding data requirements to answer query\",\n",
      "    \"type\": \"object\",\n",
      "    \"properties\": {\n",
      "        \"original_query\": {\n",
      "            \"title\": \"Original Query\",\n",
      "            \"description\": \"The original query\",\n",
      "            \"type\": \"string\"\n",
      "        },\n",
      "        \"sub_question_1\": {\n",
      "            \"title\": \"Sub Question 1\",\n",
      "            \"description\": \"Sub question 1: What are the notable achievements of Gene Kelly as a performer and director?\",\n",
      "            \"type\": \"string\"\n",
      "        },\n",
      "        \"sub_question_2\": {\n",
      "            \"title\": \"Sub Question 2\",\n",
      "            \"description\": \"Sub question 2: What are the notable achievements of Yannis Smaragdis as a filmmaker?\",\n",
      "            \"type\": \"string\"\n",
      "        }\n",
      "    }\n",
      "}\n",
      "\n",
      "## Sub Tasks\n",
      "original_query:\n",
      "Extract this data field. -> The original query\n",
      "deps: []\n",
      "\n",
      "\n",
      "sub_question_1:\n",
      "Extract this data field. -> Sub question 1: What are the notable achievements of Gene Kelly as a performer and director?\n",
      "deps: []\n",
      "\n",
      "\n",
      "sub_question_2:\n",
      "Extract this data field. -> Sub question 2: What are the notable achievements of Yannis Smaragdis as a filmmaker?\n",
      "deps: []\n",
      "\n",
      "\n",
      "merge_data_extractions:\n",
      "Use the provided data to fill in the StructuredContext data class. -> A StructuredContext object.\n",
      "deps: ['original_query', 'sub_question_1', 'sub_question_2']\n",
      "\n",
      "\n",
      "query_response_tasks:\n",
      "Who is more than just a film director, Gene Kelly or Yannis Smaragdis? -> Response to the query.\n",
      "deps: ['original_query', 'sub_question_1', 'sub_question_2', 'merge_data_extractions']\n",
      "\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'9b5607b4-26fd-44ee-8f82-b86d136e5e79'"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "agent.create_plan(\n",
    "    input=\"Who is more than just a film director, Gene Kelly or Yannis Smaragdis?\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a75087e9-8227-47f0-9b54-5eeeb758aba7",
   "metadata": {},
   "source": [
    "![Divider Image](https://d3ddy8balm3goa.cloudfront.net/mlops-rag-workshop/divider-2.excalidraw.svg)\n",
    "![Slide TwentyThree](https://d3ddy8balm3goa.cloudfront.net/genai-philippines/v1/23.svg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30cbac4d-170c-4a28-8211-92851f107e1d",
   "metadata": {},
   "source": [
    "![Divider Image](https://d3ddy8balm3goa.cloudfront.net/mlops-rag-workshop/divider-2.excalidraw.svg)\n",
    "![Slide TwentyTwo](https://d3ddy8balm3goa.cloudfront.net/genai-philippines/v1/22.svg)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "genai-philippines",
   "language": "python",
   "name": "genai-philippines"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
